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Abstract — In many statistical signal processing applications, 
the estimation of nuisance parameters and parameters of interest 
is strongly linked to the resulting performance. Generally, these 
applications deal with complex data. This paper focuses on 
covariance matrix estimation problems in non-Gaussian envi- 
ronments and particularly, the A/-estimators in the context of 
elliptical distributions. Firstly, this paper extends to the complex 
case the results of Tyler in |1|. More precisely, the asymptotic 
distribution of these estimators as well as the asymptotic distribu- 
tion of any homogeneous function of degree of the M-estimates 
are derived. On the other hand, we show the improvement of 
such results on two applications: DOA (directions of arrival) 
estimation using the MUSIC (Multiple Signal Classification) 
algorithm and adaptive radar detection based on the ANMF 
(Adaptive Normalized Matched Filter) test. 

Index Terms — Covariance matrix estimation, robust estima- 
tion, elliptical distributions, Complex M-estimators. 

I. Introduction 

Many signal processing applications require the knowledge 
of the data covariance matrix. The most often used estimator 
is the well-known Sample Covariance Matrix (SCM) which 
is the Maximum Likelihood (ML) estimator for Gaussian 
data. However, the SCM suffers from major drawbacks. When 
the data turn out to be non-Gaussian, as for instance in 
adaptive radar and sonar processing Q, the performance 
involved by the SCM can be strongly degraded. Indeed, this 
is the case in impulsive noise contexts and in the presence 
of outliers as shown in 0. To overcome these problems, 
there has been an intense research activity in robust estimation 
theory in the statistical community these last decades H, 
1 5 1, [6|. Among several solutions, the so-called Af-estimators 
originally introduced by Huber [7] and investigated in the 
seminal work of Maronna |8|, have imposed themselves as an 
appealing alternative to the classical SCM. They have been 
introduced within the framework of elliptical distributions. 
Elliptical distributions, originally introduced by Kelker in |9), 
encompass a large number of well-known distributions as for 
instance the Gaussian distribution, or the multivariate Student 
(or t) distribution. They may also be used to model heavy 
tailed distributions by means of the K-distribution, as may be 
met for instance in adaptive radar with impulsive clutter 1 10 1, 
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ifTTIl . Ifl2l . M-estimators of the covariance matrix are however 
seldom used in the signal processing community. Only a 
limited case, the Tyler's estimator lfl3ll also called the Fixed 
Point Estimator [14| has been widely used as an alternative to 
the SCM for radar applications. Concerning the M-estimators, 
notable exceptions are the recent papers by Ollila fl5l . Ifl6l . 
IfTTIl . 0~8), lfl9l who advocates their use in several applications 
such as array processing. The M-estimators have also been 
studied in the case of large datasets, where the dimension of 
the data is of the same order as the dimension of the sample 

Cl- 
One possible reason for this lack of interest is that their sta- 
tistical properties are not well-known in the signal processing 
community, as opposed to the Wishart distribution of the SCM 
in the Gaussian context. They have been studied by Tyler l2Tll 
in the real case. However, in signal processing applications, 
data are usually complex and the purpose of this paper is to 
derive the asymptotic distribution of complex M-estimators 
in the framework of elliptically distributed data. This result is 
also provided in lfT31l but without proof. We will also extend 
to the complex case, a property initially derived by Tyler in 
(TJ: we show that in the complex elliptical distributions con- 
text, the asymptotic distribution of any positive homogeneous 
functional of degree of estimates such as M -estimates and 
the SCM, is the same up to a scale factor. This result, useful 
for applications, extends the one proposed in fl31 . Thus, for a 
Gaussian context and for signal processing applications which 
only need the covariance matrix up to a scale factor, for ex- 
ample Direction-of- Arrival (DOA) estimation or adaptive radar 
detection, the parameter estimated has the same mean square 
error when estimated with the SCM or with an A/-estimator 
with a few more data (depending on <ti). Moreover, when the 
context is non-Gaussian or contains outliers, the performance 
obtained with A/-estimators is scarcely influenced while it is 
unreliable and possibly completely damaged with the SCM as 
shown for instance in Q. We illustrate this effect using the 
MUSIC method and the Adaptive Normalized Matched Filter 
(ANMF) test introduced by Kraut and Scharf |22l. Il23l. It is 
also illustrated by Ollila in lfl6l . for MVDR beamforming. 
This paper is organized as follows. Section [TT] introduces the 
required background and Section [III] the known properties of 
real M -estimators. Then Section ITVl provides our contribution 
about the estimators asymptotic distribution. Eventually, in 
Section [V] simulations validate the theoretical analysis and 
Section [VI] concludes this work. 

Vectors (resp. matrices) are denoted by bold-faced lowercase 
letters (resp. uppercase letters). *, T and H respectively 
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represent the conjugate, the transpose and the Hermitian 
operator. ~ means "distributed as", = stands for "shares the 
same distribution as", — > denotes convergence in distribution 
and ® denotes the Kronecker product, vec is the operator 
which transforms a matrix m x n into a vector of lenth ran, 
concatenating its n columns into a single column. Moreover, 
I m is the m x m identity matrix, m ,p the to x p matrix of 

m 

zeros, J m 2 = J,-j (g) Jjj where J« is the m x m matrix 

with a one in the (i, i) position and zeros elsewhere and 
K is the commutation matrix which transforms vec(A) into 
vec(A T ). Eventually, im(y) represents the imaginary part of 
the complex vector y and Re(y) its real part. 

II. Background 

A. Elliptical symmetric distribution 

Let z be a m-dimensional real (resp. complex circular) ran- 
dom vector. The vector z has a real (resp. complex) elliptical 
symmetric distribution if its probability density function (PDF) 
can be written as 

. 9z (z) = |A|- 1 /2/ lz (( z _ /x) T A -l (z _ /x)); 

in the real case, 

. 9z (z) = |A|-^ z ((z- At ) ff A- 1 (z - M )), ,0 
in the complex case, 

where h z : [0, oo) — !• [0, oo) is any function such that (fTJ 
defines a PDF, fi is the statistical mean and A is a scatter 
matrix. The scatter matrix A reflects the structure of the 
covariance matrix of z, i.e. the covariance matrix is equal to 
A up to a scale factor. This real (resp. complex) elliptically 
symmetric distribution will be denoted by £(fi, A, h z ) (resp. 
C£ (/J,, A, h z )). One can notice that the Gaussian distribution 
is a particular case of elliptical distributions. A survey on 
complex elliptical distributions can be found in |[T31 . 

In this paper, we will assume that = m .i. Without loss 
of generality, the scatter matrix will be taken to be equal to 
the covariance matrix when the latter exists. Indeed, when the 
second moment of the distribution is finite, function h z in 
(Q~|l can always be defined such that this equality holds. If the 
distribution of the data has a none finite second-order moment, 
then we will simply consider the scatter matrix estimator. 

B. Generalized Complex Normal distribution 

As written before, the Gaussian distribution is a particular 
case of elliptical symmetric distributions. However, in the 
complex framework, it is true only for circular Gaussian 
random vectors. We now present the generalization of this 
distribution as presented by Van den Bos in lf24l . 

Let z = x+jy be a m-dimensional complex random vector. 
The vector z is said to have a generalized complex normal 
distribution if and only if v = (x T , y T ) T 6 K 2m has a normal 
distribution. This generalized complex normal distribution will 
be denoted by QCAf(fi, X, fi) where fi is the mean, X = 
E[(z — /i)(z — fJ.) H ] the covariance matrix, and f2 = E[(z — 
/i)(z — fx) T ] the pseudo-covariance matrix. 



C. M-estimators of the scatter matrix 

Let (zi,...,zjv) be an A^-sample of m-dimensional real 
(resp. complex circular) independent vectors with z, ~ 
£(0 m ,i, A, h z ) (resp. z; ~ C£(0 m<1 , A, i = 1,.,.,N. 
The real (resp. complex) M-estimator of A is defined as the 
solution of the following equation 

m = - j> (z^m-^) zx- (2) 

71=1 

where the symbol ' stands for T in the real case and for H in 
the complex one. 

Af -estimators have first been studied in the real case, defined 
as solution of © with real samples. Existence and uniqueness 
of the solution of (O has been shown in the real case, provided 
function u satisfies a set of general assumptions stated by 
Maronna in (8j. These conditions have been extended to the 
complex case by Ollila in ifTTl . They are recalled here below 
in the case where /x = m ,i: 

- u is non-negative, non increasing, and continuous on 
[0,oo). 

- Let ip( s ) = su(s) and K = sup s>0 ip(s). m < K < oo, 
ip is increasing, and strictly increasing on the interval 
where ip < K. 

- Let Pn(-) denote the empirical distribution of 
(zi, zjy). There exists a > such that for every 
hyperplane S, dim(S) < m - 1, P N (S) < 1 - f - a. 
This assumption can be strongly relaxed as shown in 
ED, H26). 

Let us now consider the following equation, which is roughly 
speaking the limit of (f2| when N tends to infinity: 

M = E [tt(z'M _1 z) zz'] , (3) 

where z ~ £(0 mj i, A, h z ) (resp. C£(O mt i, A, h z )) and where 
the symbol ' stands for T in the real case and for H in the 
complex one. 

Then, under the above conditions, it has been shown for the 
real case in EH, that: 

- Equation (01 (resp. (f2])) admits a unique solution M (resp. 
M) and 

M = a' 1 A, (4) 

where a is the solution of i?[^(CT|t| 2 )] = to, 
where t - £(0 m ,i, I TO , h z ), see e.g. @ (resp. t ~ 
C£ (0 m ,i, I m , h z )). 

- A simple iterative procedure provides M. 

- M is a consistent estimate of M. 

The extension to the complex case of previous results has been 
done in ifTSI . 

D. Wishart distribution 

The real (resp. complex) Wishart distribution W(N, A) 

N 

(resp. CW(iV, A)) is the distribution of ^^z„z^, where 

71=1 

z n are real (resp. complex circular), independent identically 
distributed (i.i.d), Gaussian with zero mean and covariance 
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matrix A. Let W 



N 



be the related SCM 



which will be also referred to, as a Wishart matrix. The 
asymptotic distribution of the Wishart matrix Wat is (see e.g. 
ED) 

ViVvec(W JV - A) JV (0 mV , (A ® A)(I m 2 + K)) 
in the real case, 

VNwec(W N - A) A GCAf (o m 2 A ,A T ® A, (A T ® A)K 
in the complex case. 

(5) 

We now introduce real M -estimators asymptotic properties 
since they are used as a basis for the extension to the complex 
case. 

III. Real M-estimators properties 

A. Asymptotic distribution of the real M -estimators 

Let M be a real M-estimator following Maronnas's condi- 
tions [8 1, recalled in section IlLCl The asymptotic distribution 
of M is given by Tyler in lf2Tl : 

V^Vvec(M-M) A/"(O m ,i,II) , (6) 

where II = CTi(I m 2 + K)(M<g)M) + cr 2 vec(M)vec(M) T , a x 
and o~2 are given by ( l2T1 ): 



0"2 



with 



ai(m + 2) 2 (2a 2 + to) -2 , 

2oi(a2-l) r . . 

( fll ~ V ~ 7^ 1 \2 [™ + {m + 4 a 2 

(2a 2 + my 



a t = [TO(TO + 2)]- 1 £;[V' 2 (a|t| 2 )] , 
a a = m- 1 B[cr|t|V(o-|t| 2 )], 



(7) 



and a is given in equation (|4). 

B. An important property of real M-estimators 

Let V be a fixed symmetric positive-definite matrix and Vat 
a sequence of symmetric positive definite random matrices of 
order to which satisfies 

It , d 



7Vvec(V 



N 



M O r , 



(8) 



where S = ^i(I m 2 + K)(V ® V) + ( [/ 2 vec(V)vec(V) T , m 
and [i2 are any real numbers such that S is a positive matrix. 

Let H(V) be a r-dimensional multivariate function on the 
set of to x to positive-definite symmetric matrices with con- 
tinuous first partial derivatives and such as H(V) = H(aV) 
for all a > 0. Then under conditions ([8]), Tyler has shown in 
0] Theorem 1, that 

N(H(V N )-H(V)) -A 

Af(0 r , 1 ,2 Ml tf / (V)(V®V)ff / (V) T ), (9) 



0- 



where ^(V).i(^)(I m2+ J r 

By noticing that, in a Gaussian context the SCM 
satisfies fii = 1 and /i 2 = (equation ©) and that real 



M-estimators verify /i X = u\ and /i 2 = <r 2 (equation 
©), T yler's theorem shows that </N(H(W N ) - H(A)) 
and yjN/<7i{H(Nl) — H{A)) share the same asymptotic 
distribution. 

In practice, H(.) may be a function which associates 
a parameter of interest to a covariance matrix. This 
scale-invariant property has also been exploited in [?]. 
The concerned signal processing applications are those 
in which multiplying the covariance matrix by a positive 
scalar does not change the result. This is the case for 
instance for the MUSIC method in which the estimated 
parameters are the signals DOA. An other example is given 
by adaptive radar processing in which the parameter is 
the ANMF test statistic E2), EH. Here, H is defined by: 



H 



Ip^m-VI 2 



m 4 Him) = - 

(p"M-iy)(y"M-V) 
The aim of the next section is to extend those results to 
the complex case, which is the frequently met framework for 
most signal processing applications. 

IV. Main results in complex case 
A. Asymptotic distribution of the complex M-estimator 

Let (zi, Zjy) be an TV-sample of m-dimensional complex 
independent vectors with z„ ~ C£ (0 TO) i, A, h z ), n = 1, N. 
We consider the complex M -estimator Mp which verifies 
equation (0, and we denote Mp the solution of (01. 

Theorem IV.l The asymptotic distribution of Mp is given by 

Vn^c(M c - M ) A QCN (0 m 2 A , E, ti) , (10) 

where S and f2 are defined by 

£ = criMg ® M c + cr 2 vec(M c )vec(M c ) H , 
ft = cri(Mg ® M C )K + cr 2 vec(M c )vec(M c ) T 



with 



a i = a\{m + l) 2 (a 2 + to)" 



<T 2 



(oi - 1) 



2ai(a 2 - 1) 
(2a 2 + 2to) 2 



(11) 



[2m + (2m + 4)a 2 

(12) 



and 



oi = Km + ljj-^^Htl 2 )] , 

a2 =TO- 1 i?[ ( 7|t| 2 ^(a|t| 2 )], 

where a is the solution of E[ip(a\t\ 2 )] — to, where t ~ 

This result is also given in [15| with others assumptions but 
without proof. 



B. Proof of Theorem \IV.1\ 

1) Notations: Let us first introduce the following linear 
one-to-one transformation of a Hermitian to x to matrix A 
into a real symmetric 2to x 2to matrix: 



J[ ' 2Uto(A) Re(A) 



(13) 
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The inverse transformation is given by A = g H /(A)g where 
g r = (I TO , — j'I m ). Function / has some useful properties. 
Let u„ and v„ be the following 2 m vectors: 

u„ = (i?e(z„) T ,/m(z„) T ) T 

v n = (-Im(z n ) T ,Re(z n ) T ) T , (i4) 

which are both distributed according to £ (02 m ,i, A R , g z ) 
where A R = /(A). 

Then, it may be shown that 

J(A- 1 ) = i/(A)- 1 
/(z„z^ ) = - {u n ul + v„v£) 



Irn I. 



, one has v n = T r u n and u n 



(15) 



Let T r = 

T r T v„. 

Let us also introduce 

M R = f(M c ), 
M R = /(Mo). 

It is easy to show that equation (|2]i defining the complex 
Af-estimator Mp, is equivalent to the following equation 
involving M^: 

1 N — 

n=l 

u r (v,^M fl 1 v„) v„v^) , (16) 

where u r (s) — u(s/2). Roughly speaking, equation ([Tol l 
defines a real Af-estimator involving the 2N real samples u ra 
and v„. 

Let M u and M„ be respectively the two M-estimators 
defined by 



^ 1 N - 



n=l 
N 



(17) 



M - = nJ2 Ur ( v «^ lv ") v " v 



and let M u , M ( be the associated solutions of 

M U = E [u r (ujM- 1 ^) Ul uf] , 
M v = E [u r (vf M-^i) vivf ] . 

By applying T r on equation (jTTJ, one obtains 

M» = T r M u T r T . 
Moreover, since v„ has the same distribution as u„ 
M u = M, = T r M u T r T 
2) An intermediate result: 

Lemma IV.l and -(M„ + M„) have the same Gaussian 
asymptotic distribution. 

Proof: See appendix lAl ■ 



(18) 
(19) 



3) End of proof of theorem \IV.1\ By using^ equation (fl3T > 
and the inverse off, one obtains Mc = g ff M^g. From the 
Lemma Hv. II vecfM/A has a normal distribution. It follows 
that vec(Mc) has a generalized complex normal distribution. 

Given the property vec(ABC) = (C T ® A)vec(B) where 
A, B, C are 3 matrices, and using the fact that Mc* = 
g H lsA R g, one has 

S = NE[\ee(M c - M c )vec(M c - M C ) H ] 

= (g T ® S H ) E[Nvec(M R - M R )vec(M R - M R ) H ] 

(z T ®g H ) H - 

(20) 

Using lemma HvTl and the equalities (TT8l and H9i . equation 
(l20l i gives 



(g T ® g H )NE 



vec -(M„ + T r M u T r 



vec -(M u + T r M„T r 



M„ 



(g' J 



= (g T ® g ff )(W + T r ® T r )n u (I 4m 2 + T r ® T r ) H 
(g T ®g ff ) H 

= (g T ®g ff )n„(g T ®g ff ) ff , 

(21) 

by using U u the asymptotic covariance of M u and the 
equalities g T T r = — jg T and g ff T r = jg H . 

Using the expression given in ©, and taking into account 
that the u„ are 2m-dimensional vectors, we have 

n„ = <ri(I 4m 2 + K)(M U ® M u ) + cr2vec(M u )vec(M u ) T 

(22) 

where <ji and a 2 will be specified later. 

A consequence of lemma HvTI is that = M u . Indeed, 
from the definition of M R , one has 

M R = \- (E [u r (u T M^u) uu T ] + E [u r (v T M^v) vv T ]) 

The first term of the right hand side is the definition of M u 
while the second one is the one of M„. Then, as M u = M„, 
one has M R = M„. 

Therefore, Mc = g^M„g, which leads to 

S = 01 (Mg <g> M c ) + o- 2 vec(M c )vec(Mc) H . (23) 

Now let us turn to the o\ and er 2 coefficients. Using ©, 
one has 

(7i = ai(2m + 2) 2 (2a 2 + 2m)" 2 , 
2oi(a 2 - 1) 



<r 2 = a 2 



(01 - 1) - 



(2a 2 + 2m) 2 



[2m + (2m + 4)a 2 ] 



a x = [2m(2m + 2)]" 1 E [^ 2 (a|s| 2 )] 
{ a 2 = (2m)- 1 £;[a|s| 2 <(a|s| 2 )], 



(24) 



where ip r (s) = su r (s), s ~ £ (0 2mj i, I 2m , /i z ) and ct is the 
solution of 

^[^(crlsl 2 )] = 2m. (25) 
Since ip r (s) — 2ip(s/2), equation ( f25T > is equivalent to 



E 



* 2 N ' 



(26) 
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Moreover let t ~ C£(O m> i, I m , h z ). Then |t| 2 has the same 
distribution as |s| 2 /2 so that (l23T > and d26l i are also equivalent 
to 

E [VMt| 2 )] - to (27) 

We finally obtain the expression of S. 

a) Asymptotic pseudo-covariance matrix: CI is defined 

as 

O = iV£[vec(M c - M c )vec(M c - M C ) T ] (28) 
Using the commutation matrix K, one has 

Kvec(M c - M c ) = vec(Mg - Mg) 



(29) 



(30) 



= vec(M c - M c )* 
since Mc is Hermitian. Thus one can write 

vec(M c -M c ) T = [vec(M c -M c )*p, 
= vec(M c - M c ) ff K, 

where K H = K. 

Therefore, Jl = £K, which leads to the result of theorem 
II V. 1 1 after a few derivations, and concludes the proof. 

In the following part, we extend the result of section IIII-BI 
to the complex case. 

C. An important property of complex M -estimators 
Theorem IV.2 

• Let M be a fixed Hermitian positive-definite matrix and 
M a sequence of Hermitian positive-definite random 
matrix estimates of order m which satisfies 

d 



N vec(M-M) -^■gCM(0 m 2 tl ,'E M ,n 



M 



with 



viM T ® M + ^ 2 vec(M)vec(M) ff , 



(31) 



(32) 



vi (M T ® M)K + z/ 2 vec(M)vec(M) T , 
where V\ and v 2 are any real numbers. 

Let iJ(M) = (hi, h r ) T be a r-dimensional multivari- 
ate function on the set of to x m complex Hermitian 
positive-definite matrices, possessing continuous first par- 
tial derivatives and such as H(M.) = iJ(aM) for all 
a>0. 



Then, 

Vn (h(m) - H(M)) -46CJ\f(0 ril ,S Hi n H ) 

where £# and fin are defined as 

E H = v 1 H'(M)(M T ® M)H'(M) H , 
Cl H = v 1 H'(M)(M T ® M)KiT(M) T , 



and H'(M) = 
vec(M) = (rrii 



dH(M) dh % 

= (Ki) Wlth h a = 



dvec(M) 



9to, 



(33) 



(34) 



where 



Proof: One can first notice that ZT(M)vec(M) = r l . 
Indeed, since H(NL) = H(aM.) for all a > 0, the subspace 
generated by the vector vec(M) is an iso-H region. Therefore, 



H'(M) which can be seen as a gradient of H, is orthogonal 
to vec(M). 

A first order approximation of H (M) gives 

H(M.) ~ H(M) + //"'(M)vec(M — M), (35) 

Thus one has, 

S H =NE (h(M) — H(M) \ (h(M) — H(M.)j 

= H'(M)E [A^vec(M - M)vec(M - M) H 1 H'{M) H 
= ir(M)£ M #'(M) ff 

= H'(M) (v 1 M T <g> M + ^ 2 vec(M)vec(M) ff ) H'{M) H 
= H'(M) (i/iM T ® M) H'(M) H . 

(36) 

The proof is similar for ■ 
Similarly to the real case, when the data have a complex 
Gaussian distribution, the SCM is a complex Wishart matrix. 
Moreover, the SCM estimator verifies the conditions of the 
theorem and its coefficients (pi,p2) are equal to (1,0). 
Complex normalized M-estimators also verify the conditions 
of the theorem with (pi,p 2 ) = (o"i,cr 2 ). Thus they have 
the same asymptotic distribution as the complex normalized 
Wishart matrix, up to a scale factor a% depending on the 
considered M-estimator. The same conclusion holds for 
the Fixed Point Estimator JT3), [?] since it verifies the 
assumptions of theorem IIV.2I (see ||28l for its asymptotic 
distribution). 



V. Simulations 

The results of this paper are illustrated using the complex 
analogue of Huber's M-estimator as described in ifTTl . The 
corresponding weight function it(.) of equation © is defined 

by 

u(s) = imin(l,fc 2 /s), (37) 

where k 2 and (3 depend on a single parameter < q < 1, 
according to 

(38) 



q = F 2m (2k 2 
P = F 2m+2 (2k 2 ) + k 



2> ,2 1 - g 
TO 



(39) 



where F m (.) is the cumulative distribution function of a x 2 
distribution with to degrees of freedom. Thus Huber estimate 
is the solution of 



— 1 N 



<k 2 



+ k 2 



z,z 



H 



(40) 



which can be rewritten 

N 



<fc 2 



i=l 



HtTX-1 z f M n],h z i> k2 

Z i M Hub^ * 



(41) 
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where 1 is the indicator function. 

The first summation corresponds to unweighted data which 
are treated as in the SCM; the second one is associated to 
normalized data treated as outliers. In a complex Gaussian 
context and when TV tends to infinity, it may be shown that 
the proportion of data treated with the SCM is equal to q. 
Moreover the choice of k 2 and (3 according to d38l l and (f39b . 
leads to a consistant Af -estimator of the covariance matrix 
(a = 1 in equation 

In the following simulations, q = 0.75. 

A. Asymptotic performance of DOA estimated by the MUSIC 
method, with the SCM and Huber's M -estimator 

Now let us turn to theorem [IV2] To illustrate our result, we 
consider a simulation using the Multiple Signal Classification 
(MUSIC) method, which estimates the Directions Of Arrival 
(DOAs) of a signal. We consider in this paper a single signal 
to detect. However, the multi-sources case can be similarly 
analyzed. Under this assumption, let us define H(M) the 
estimated DoAs obtained from the MUSIC pseudo-spectrum: 
H(M) = 6. 

A m — 3 uniform linear array (ULA) with half wavelength 
sensors spacing is used, which receives a Gaussian station- 
nary narrowband signal with DOA 20°. The array output is 
corrupted by an additive noise which is firstly spatially white 
Gaussian and secondly K-distributed with shape parameter 0. 1 . 
Moreover, the SNR per sensor is 5dB and the N snapshots 
are assumed to be independent. The MUSIC method uses the 
estimation of the covariance matrix with the N snapshots and 
here, the employed covariance matrix estimators are the SCM 
and the complex analogue of Huber's Af-estimator as defined 
in equation (f4Tb . 

Figure Q] depicts the Root Mean Square Error (RMSE) in 
degrees, of the DOA estimated with N data for the SCM 
and for Huber's estimate, when the additive noise is white 
Gaussian. The RMSE of the DOA estimated with o\ N data 
with Huber's estimate is also represented. We observe that 
for N large enough (N > 40), this curve and the SCM one 
overlap, as expected from theorem dIV.lt . In this example, 
eri = 1.067. 
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Fig. 1 . One source DOA RMSE (m = 3 antennas) for Huber's estimate and 
the SCM, for spatially white Gaussian additive noise. 



Figure depicts the RMSE of the DOA estimated with N 
data for the SCM and for Huber's estimate, when the additive 
noise is K-distributed with shape parameter 0.1. A shape 
parameter close to 1 (> 0.9) indicates a distribution close 
to the Gaussian distribution whereas it indicates an impulsive 
noise when the parameter is close to (< 0.1). Thus, the noise 
being quite impulsive in our example, we observe that the 
RMSE of Huber's Af -estimator is smaller than the SCM, the 
latter giving worse results than in the Gaussian case. It points 
out the fact that the SCM gives poor results as soon as the 
context is far from a Gaussian environment whereas Huber's 
A/-estimator is more robust and much more interesting in that 
case. 




150 200 250 300 350 

Number of snapshots 



Fig. 2. One source DOA RMSE (m = 3 antennas) for Huber's estimate and 
the SCM, for K-distributed additive noise with a shape parameter v = 0.1. 



B. Asymptotic performance of the ANMF test with the SCM 
and Huber's M -estimator 



Let us give a second illustration of theorem IIV.2I We 
consider an adaptive radar receiving a vector y of length m. 
The estimated covariance matrix of the environment is M and 
we try to detect signals of steering vector p. This steering 
vector defines the DOA and speed of the target, using the 
Doppler frequency. The ANMF test statistics |29l is 



A(M|y) 



Ip^m-VI 2 



(p"M-iy)(y"M-iy) 



(42) 



Firstly, we have considered a Gaussian context and com- 
puted A(M|y). In figure [3] the vertical scale represents the 
variance of A obtained with the SCM and the complex ana- 
logue of Huber's Af-estimator defined in (0. The horizontal 
scale represents the number of samples used to estimate the 
covariance matrix. A third curve represents the variance of A 
for <j\N data. As one can see, it overlaps the SCM's curve, 
illustrating theorem|lV2] The coefficient o\ is equal to 1.067. 
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Fig. 3. Variance on the ANMF detector for Huber's estimate and the SCM 
estimate, with spatially white Gaussian additive noise. 



Secondly, we have considered a K-distributed environment, 
with shape parameter firstly equal to 0.1 and then 0.01 for a 
more impulsive noise. The figure [4] which scales are the same 
as in figure |3] brings once again to our minds that the SCM is 
not robust in a non-Gaussian context contrary to Huber's M- 
estimator. Indeed, the more the noise differs from a Gaussian 
noise, the more the detector's variance is deteriorated in that 
case while it still gives good results with Huber's A/-estimator. 




(A H , lh ), f = 0.01 
>'( \scm). " = 01 

(\n.i, f = o.i 
i Vscm), "-"I 



300 400 500 000 700 

Number of snapshots N 
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Fig. 4. Variance on the ANMF detector with the Huber's estimate and the 
SCM, for K-distributed additive noise with various shape parameters v = 0.01 
and nu = 0.1. 



VI. Conclusion 

In this paper we have analyzed the statistical properties of 
complex M -estimators of the scatter matrix in the framework 
of complex elliptically distributed data. Firstly, using existing 
results for real Af-estimators, we have derived the asymptotic 
covariance in the complex case. Simulations have checked 
that when the number of samples increases, the A/-estimator 
covariance tends to its theoretical asymptotic value. Secondly, 
we have extended an interesting property of real A/-estimators 
to the complex case. This property states that the asymptotic 
distributions of any homogeneous function of degree zero 
of M-estimates and Wishart matrices, are the same up to a 
scale factor. This result has many potential applications in 
performance analysis of array processing algorithms based on 
M-estimates of the covariance matrix. 



Proof: Lemma H V. 1 1 



A. Asymptotic behavior of M M and M„ 
Let us set 

. M u = Mi /2 Mi /2 , 

. R„ = M^ 1/2 M„M- 1/2 and 

. k n = m.;; 1/2 u„. 

Since R u is a consistent estimate of I m , when N — > oo, we 
have R u = l2 m + AR M , considering AR U small. Thus we 
have 



(u£M- x u n ) = u^R^k,, 

= u (k^(I 2m + AR u ) _1 k„ 



A first order expansion of R u gives (l2 m + AR U ) 1 
l2 m — AR„ which leads to 



(k^R-H 



u (||k r , 
u (Ilk*, 



k£AR„k„) 



(43) 



/(||k„|| 2 ) k^AR„k 



a n + ^nk„ AR u k„, 



with a n = u (||k„|| 2 ) and b r , 
(fTTI i. we obtain 



i' (||k„|| 2 ). From equation 



1 N 1 N 

I 2m +AR„ = — ^ a n k n kl+—J2 b n (k^AR u k„) k„k^. 

n— 1 n— 1 

Since vec(k„k^) = (k„ ® k„) and k^AR u k„ = 
vec (k^AR„k„) = (k„ g) k n ) T vec(AR u ), one has the 
following equation: 

JV 



1 

vec(I 2m ) + vec(AR„) = a " k ™ ® k 



n=l 



1 

N 



N 



■J2 & «(k„ ® k„) T vec(AR n )(k„ ® k„). (44) 

n=l 

This leads to 



1 N \ 
I4m2 - j-f^2b n (k n ® k n )(k„ ® k n ) T J vec(AR„) 

71=1 / 

1 N 

n=l 

1 N \ 

6 n (k„ (g> k„)(k„ ® k„) T J . Then the 

n=l / 

is equivalent to 

( 1 N \ 
vec(ARu) = a.^ I — ^2 a «( k " ® k„) - vec(I 2m ) J . 



Let us denote 
previous equation is equivalent to 



In (l46*l l we have 

. cxn ^ a = (I 4m 2 - E[b (k <g> k)(k <g> k) T ]) where k . 
£(0 2 m,i,cr r I 2m ,3 z ) and b = -u' (||k|| 2 ) 



1 N 

1 v-^ , n as ■. , , Then using the vec operator, this equation leads to 

— 2^a„(k„ ® k„) -> £[ak®k] 

^Kl|k|| 2 H)k^k] = vec(E[.(||kf||)kk-]) = vec(I 2m ) + vec(AR,) ^ 

vec (I 2m ) using © with M u and replacing u„ by - 1 Sr„i rl9l ^j. 1 Sr„r r ^ l9l r r ^ 
jyj-i/2^ — 27V ^-^ ™ ™ 59 ™ + ~2]\[ 2-*/ n ±rKn 69 lrK « 



Now let us denote wat = 1 w 

/t;/ 1 ^ /, »i 1 / T \\ w , + — Vfe„(k„(g)k„) T vec(AR i? ,)(k n (g)k n ) 

V-/V [ — 2^ a„(k„ ® k„) - vec(I 2m ) 1 . We have 2N ^-J 



d 1 N 

wjv — > w, where w follows a zero mean Gaussian + — V 6„(T r k„ ® T r k„) T vec(AR fi )(T r k„ ® T r k„). 

distribution. 2 AT 



ra=l 



Consequently, the Slutsky theorem gives 

\/iVvec(AR u ) = arfw N -A a^w. (47) This is equivalent to 

Moreover, one can notice that / \ JL^ 

(My 2 ®My 2 )vec(AR u ) = vec(M*/ AR^M^ ) ^ 2N ^ 

= vec(M„ - M„), i N 

,-, ■ , • • • urn — V6„(T r (»T r )(k n ®k„)(k„(8)k rl ) T (T r (8)T r ) 3 

which gives, taking into account equation (147 l i. ^jV 



v^vec(M u - M u ) A (M^ 2 ® M^a^w. (48) vec(AR fi ) 

1 N 

Using equation dTHJ, we also have = — ^ a „ (k„ ® k Tl 

1 



7Vvec(M„ - M„) = VN(T r ® T r )vec(M u - M u ) 

-A (T r T r )(My 2 ® Mfja-'w. (49) + 2A^ a " (Tr Tr)(k " kn) " vec (W 



n=l 
N 



n=l 



where T r =(Y m n Im )- which leads to 

^-1 / 1 . , 

B. Asymptotic behavior of M R vec(AR R ) = a N I 7^2^ a "( k " ® k ") 

y , ft , 1/2S"; ax-1/2 , T , , \ n=l 



Let us denote R R = M fl 1/2 MRM R 1/2 . Since M R = M u , 



N 



one has k„ = M R 1/2 u n . + J_ <T a ^ Tr ® Tr)(kn g kn) _ vec(l2n 

For all matrices of the form /(A), T r /(A) = /(A)T r . 2N tZi 

Therefore, since M R = f(M c ), T r M R = M R T r . One 



has M* = T SM R T r = (T/MfT r ) (t/M^ T r ). *„« w= L-if l ,(k. 8 k.)(l, 8 k/ 

Therefore mJ/ 2 = T r T M^ /2 T r and T r Mjj' 2 = M)/ 2 T r . \ ^ V „=i 

This leads to T r k„ = T r M fl 1/2 u„ = M fl 1 / 2 v„. - — El, b n (T r ® T r )(k„ ® k„)(k„ ® k„f (T r ® T r ) T ^ 

When — >• oo, since R^ is a consistent estimate of I 2m , 2N j 

Ri? = hm + ARr, with ARij small. Similarly to the first Using previous notation wjv, we obtain 
part of the proof on has, 

U (u^M^u n ) = u^M^Vn) vec(AR fl ) = ^=S^ (I 4m a + (T r ® T r )) Wjv 

u {^ g "n^' R k nJ Q ne can n0 ];i ce ma ); aN ±^ q, since the T r k„ have the 

= a„ + fonk^ARflkn. same distribution as the k„. Moreover a N (T r ®T r ) = (T r <E) 

Thus, deriving from equation (dSJ we obtain T r) "at and (My 2 (g)My 2 )(T r ®T r ) = (T r (g)T r )(My 2 ® 



lam + AR/j 
1 

~ 2iV 



1/2 

Mi /Z ). Therefore we obtain 



i A ^ i N 



S fl " k " k " + 2ivS a " Trk " k " TrT ViVvec(M, - M fl ) A i ((M^ ® 

-i-E 6„ (k-AR fl k„) k„k- +( T r ® T r)(Ml /2 ® My>-w 



n=l 

I at This leads to the conclusion that M R shares the same asymp- 

bn (k^T r T AR fl T r k n ) T r k„k^T r T . ^ distribution as i f m u + M„l . ■ 
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